Camera system including a proximity sensor and related methods

ABSTRACT

A camera for capturing an image comprising: an image sensor configured to generate image sensor data in response to received light; a processing resource configured to process the image sensor data to obtain image data and communication data, wherein obtaining the communication data comprises performing a demodulation process in respect of at least part of the image sensor data, wherein the processing resource is further configured to transmit the communication data and the image data to at least one further processing resource.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is continuation of U.S. patent application Ser. No.16/087,202, filed Sep. 21, 2018, which itself is a 35 U.S.C. § 371national stage application of PCT International Application No.PCT/GB2017/050792, filed on Mar. 21, 2017, which claims priority fromGreat Britain Patent Application No. 1605142.7, filed on Mar. 25, 2016,the contents of which are incorporated herein by reference in theirentireties. The above-referenced PCT International Application waspublished in the English language as International Publication No. WO2017/163054 A1 on Sep. 28, 2017.

FIELD OF THE INVENTION

The present invention relates to a camera system, for example a camerasystem that includes at least one of an image sensor, a proximitysensor, an ambient light sensor, and a flash unit.

BACKGROUND OF THE INVENTION

Camera modules have been widely adopted in smart devices, such as smartphones and tablets. In addition to front and/or back facing cameras,such devices usually have a sensor for sensing proximity signals,sensors for sensing ambient light and flash units.

Wireless optical communication offers advantages over conventional radiofrequency wireless communication, and has been implemented using avariety of devices.

SUMMARY OF THE INVENTION

In a first aspect of the invention, there is provided a camera forcapturing an image comprising: an image sensor configured to generateimage sensor data in response to received light; a processing resourceconfigured to process the image sensor data to obtain image data andcommunication data, wherein obtaining the communication data comprisesperforming a demodulation process in respect of at least part of theimage sensor data, wherein the processing resource is further configuredto transmit the communication data and the image data to at least onefurther processing resource.

The image data may comprise pixel data. The communication data maycomprise optical wireless communication data.

The image sensor and processing resource may form part of or be on asingle chip.

The at least one further processing resource may be outside the chipand/or may comprise or form part of a further chip. The at least onefurther processing resource may be within the camera, for example withinan outer housing of the camera.

The processing resource may comprise a first processor and a secondprocessor. The first processor may be configured to process the imagesensor data and the second processor may be configured to process thecommunication data. Thus, separate processors may be provided. Theprocessing of the image sensor data and the communication data maycomprise processing the image sensor data and the communication datasuch that it is in a form to be transmitted via an interface to thefurther processing resource. The interface may comprise a camera serialinterface.

The camera may comprise a control bus or interface for carrying controldata between the processing resource and the further processingresource, and a further bus or interface for carrying the communicationdata and/or image data between the processing resource and the furtherprocessing resource. The further bus or interface may comprise a CSI busor interface, for example a CSI-2 or CSI-3 bus or interface. The controlbus or interface may comprise an I2C bus or interface. The control busor interface may be configured to operate at a slower data transmissionrate and/or have a lower bandwidth than the further bus or interface.The communication data may be received and/or transmitted and/ordemodulate at a rate of at least 10 Mbps, optionally at least 20 Mbps,optionally at least 40 Mbps.

The further processing resource may comprise a camera processor forcontrolling operation of the camera. The camera processor may control atleast one aspect of operation of the camera, for example capturingimages, controlling operation of at least one of image exposure time,image resolution, aperture, controlling storage and/or retrieval of datafor example images in a memory, controlling interaction with a userand/or display or operation of a user interface.

The image sensor may comprise an array of sensor elements comprisingimage sensor elements for generating image signals in response toreceived light. Each image sensor element may be configured to senselight of a respective one of a plurality of different colours. The arraymay comprise communication sensor elements for generating communicationsignals in response to received light. The communication sensor elementsmay be distributed in the image sensor array in groups of one or morecommunication sensor elements. One or more, optionally each, optionallysubstantially all, of said groups may be surrounded by image sensorelements.

The image sensor elements configured to sense light of a respective oneof a plurality of different colours may be sensitive to light ofdifferent wavelength ranges. The different types of image sensorelements may have different profiles of sensitivity as function ofwavelength. The light may comprise light of any suitable part of theelectromagnetic spectrum, for example visible light or infrared light.

The array may comprise a repeating arrangement of unit cells, each unitcell may comprise at least one communication sensor element and aplurality of image sensor elements.

The unit cell may comprise three image sensor elements.

The image sensor elements of the unit cell may comprise a sensor elementfor sensing red light, a sensor element for sensing green light and asensor element for sensing blue light.

Each communication sensor element may be configured to sense light ofthe same wavelength or wavelength range. For example, each communicationsensor element may have substantially the same profile of sensitivity asfunction of wavelength. Alternatively, each communication sensor elementmay be configured to sense light of a respective one of a plurality ofdifferent wavelengths or wavelength ranges. For example, eachcommunication sensor element may have a respective one of a plurality ofdifferent profiles of sensitivity as function of wavelength.

The image sensor may comprise an image sensor array of sensor elementscomprising: image sensor elements for generating image signals inresponse to received light, wherein each image sensor element may beconfigured to sense light of a respective different colour;communication sensor elements for generating communication signals inresponse to received light, wherein the communication sensor elementsmay be arranged in one or more rows or columns extended along one ormore edges of the image sensor array.

The camera may further comprise: image signal conditioning circuitryconfigured to collect the generated image signals from the image sensorelements and condition the generated image signals into a form suitablefor the processing resource; and communication signal conditioningcircuitry to collect the generated communication image signals from thecommunication sensor elements and condition the generated communicationsignals into a form suitable for the processing resource.

The communication signal conditioning circuitry may comprise one or moreanalogue to digital convertors (ADCs) configured to sample thecommunication signals at a frequency characteristic of an optical lightcommunication signal.

The image signal conditioning circuitry may comprise at least oneanalogue to digital convertor (ADC).

The ADC(s) of the communication signal conditioning circuitry may beconfigured to operate, for example to sample signals, at a faster ratethan the ADC(s) of the image signal conditioning circuitry. The ADC(s)of the communication signal conditioning circuitry may be configured tosample the communication signal at a rate of at least 100 MHz,optionally at least 160 MHZ. The ADC(s) of the communication signalconditioning circuitry may be larger, for example have a largercross-sectional area, than the ADC(s) of the image signal conditioningcircuitry. A separate ADC of the image signal conditioning circuitry maybe provided for each image sensor element or for each sub-group of imagesensor elements, for example each unit cell. A separate ADC of thecommunication sensor element may be provided for each communicationsensor element.

The processing resource may be configured to transmit the communicationand image data over an interface in parallel or in series.

The interface may comprise a camera serial interface (CSI). Theinterface may comprise a CSI-2 or CSI-3 interface.

The camera may comprise a flash unit. The processing resource may befurther configured to provide a modulation signal encoding communicationdata to the flash unit such that the flash unit produces an opticalcommunication signal.

The flash unit may further comprise one or more flash emitters and/orone or more communication emitters. The processing resource may befurther configured to provide the modulation signal encodingcommunication data to the flash emitters and/or a flash signal to theflash emitters.

In a further aspect of the invention, which may be providedindependently, there is provided a proximity sensor configured togenerate proximity data representative of a nearby object using opticalsignals comprising:

-   -   an emitter configured to produce an outgoing optical signal;    -   a receiver configured to receive an incoming optical signal; and    -   a processing resource configured to perform at least one of a)        and b)        -   a) a demodulation process in respect of at least part of the            incoming optical signal to obtain incoming communication            data;        -   b) a modulation process to encode outgoing communication            data on at least part of the outgoing optical signal.

The processing resource may be further configured to transmit theincoming communication data to at least one further processing resourceand/or to receive the outgoing communication data from at least onefurther processing resource

The emitter, receiver and processing resource may be integrated on asingle chip.

The at least one further processing resource may be outside the chipand/or may comprise or form part of a further chip.

The proximity sensor may further comprise driving circuitry to provide amodulation signal to the emitter. The modulation signal may be based onthe outgoing communication data received from one further processor.

The driving circuitry may comprises a digital to analogue convertorconfigured to provide a modulation signal at a frequency characteristicof an optical light communication signal.

The receiver may comprise a sensor generating a communication signal inresponse to received light, and conditioning circuitry to collect thecommunication signal and condition the communication signal into a formsuitable for the processing resource.

The processing resource may be configured to transmit and/or receivecommunication data and proximity data in series over a first interface.

The processing resource may be configured to transmit and/or receivecommunication data and proximity data in parallel over a first andsecond interface.

The first interface may comprise a high speed interface.

The second interface may comprise a slower interface than the firstinterface, for example the second interface may be configured totransmit data at a lower rate than the first interface. The secondinterface may be configured to transmit some or all of the proximitydata. The first interface may be configured to transmit some or all ofthe communication data.

The first interface may comprise a camera serial interface (CSI). Thefirst interface may comprise, for example, a CSI-2 or CSI-3 interface.

The emitter and receiver may be positioned on an edge of a housing ofthe proximity sensor.

In a further aspect of the invention, which may be providedindependently, there is provided a camera system comprising:

-   -   a camera comprising:        -   an image sensor configured to generate image sensor data in            response to received light;        -   a camera processing resource configured to process the image            sensor data to obtain image data and incoming communication            data.

Obtaining the incoming communication data may comprise performing ademodulation process in respect of at least part of the image sensordata. The processing resource may be further configured to transmit theincoming communication data and the image data to at least one furtherprocessing resource. The system may further comprise a proximity sensorconfigured to generate proximity data representative of a nearby objectusing optical signals. The proximity sensor may comprise an emitterconfigured to produce an outgoing optical signal. The proximity sensormay comprise a proximity processing resource configured to perform amodulation process to encode outgoing communication data on at leastpart of the outgoing optical signal and/or to receive the outgoingcommunication data from at least one further processing resource.

The camera system may comprise an integrated camera and proximity sensormodule. Each of the components of the camera system may be within asingle housing.

The camera processing resource may be configured to transmit theincoming communication data and image data in series or in parallel overa first interface. The proximity processing resource may be configuredto receive outgoing communication data over a second interface.

The camera system may further comprise a control bus for carryingcontrol data between the camera processing resource, the proximityprocessing resource and at least one further processing resource.

In a further aspect of the invention, which may be providedindependently, there is provided an image sensor array comprising:

-   -   image sensor elements for generating image signals in response        to received light, wherein each image sensor element is        configured to sense light of a respective different colour;    -   communication sensor elements for generating electrical        communication signals in response to received light, wherein    -   the communication sensor elements are either:        -   distributed in the image sensor array in groups of one or            more communication sensor elements, wherein one or more said            groups are surrounded by image sensor elements or        -   the communication sensor elements are arranged in one or            more rows or columns extended along one or more edges of the            image sensor array.

In a further aspect of the invention, which may be providedindependently, there is provided a flash unit for use with a camera,comprising an emitter configured to produce an outgoing optical signal,wherein the flash unit is configured to receive a modulation signalencoding communication data and in response to produce the outgoingoptical signal such that it comprises or represents the communicationdata.

The outgoing optical signal may comprise a modulated opticalcommunication signal. The emitter may be further configured to receive aflash signal and to output a flash in response to the flash signal.

In a further aspect of the invention, which may be providedindependently, there is provided a flash system comprising a flash unitas claimed or described herein and a processing resource configured togenerate the modulation signal and provide the modulation signal to theemitter.

The flash unit may further comprise one or more flash emitters and oneor more communication emitters, wherein the processing resource isfurther configured to provide the modulation encoding communication datato the flash emitters and a flash signal to the flash emitters.

In further aspects of the invention there is provided a camera asclaimed or described herein or a camera system as claimed or describedherein, further comprising a proximity sensor as claimed or describedherein and/or an image sensor array as claimed or described hereinand/or a flash unit as claimed or described herein and/or a flash systemas claimed or described herein.

In a further aspect of the invention, which may be providedindependently, there is provided a method of receiving communicationdata comprising generating image sensor data in response to receivedlight, processing the image sensor data to obtain image data andcommunication data, wherein obtaining the communication data comprisesperforming a demodulation process in respect of at least part of theimage sensor data.

In a further aspect of the invention, which may be providedindependently, there is provided a method of transmitting or receivingcommunication data comprising: using an emitter of a proximity sensor toemit light that comprises or represents communication data; and/or

-   -   using a receiver of a proximity sensor to receive light that        comprises or represents communication data, generating a signal        by the receiver in response to the received light, and        processing the received signal to extract the communication        data.

In a further aspect of the invention, which may be providedindependently, there is provided a method of transmitting communicationdata comprising using a camera flash unit to emit light that comprisesor represents communication data.

Features in one aspect may be applied as features in another aspect inany appropriate combination. For example, any one of apparatus, camera,array, sensor, system or method features may be applied as any one otherof apparatus, camera, array, sensor, system or method features.

BRIEF DESCRIPTION OF THE DRAWINGS

Various aspects of the invention will now be described by way of exampleonly, and with reference to the accompanying drawings, of which:

FIG. 1 is a block diagram of an image sensor with wireless opticalcommunication capability according to an embodiment;

FIG. 2 is a representative diagram of an arrangement of sensor elementsaccording to an embodiment;

FIG. 3 is a representative diagram of an arrangement of sensor elementsaccording to an embodiment;

FIG. 4 is a schematic diagram of a mode of operation of an interface ofan image sensor according to an embodiment;

FIG. 5 is a schematic diagram of an alternative mode of operation of aninterface of an image sensor according to an embodiment;

FIG. 6 is a block schematic diagram of the image sensor with anadditional flash unit according to an embodiment;

FIG. 7 is a block diagram of an image sensor with wireless opticalcommunication capability according to an embodiment;

FIG. 8 is a schematic diagram of a mode of operation of an interface ofa proximity sensor according to an embodiment;

FIG. 9 is a schematic diagram of an alternative mode of operation of aninterface of a proximity sensor according to an embodiment;

FIG. 10 is a top view of a proximity sensor according to an embodiment,and

FIG. 11 is a block schematic diagram of a camera and proximity sensormodule according to an embodiment.

DETAILED DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram of an image sensor 10. The image sensor 10has the following components: an image sensor array 12, an analoguefront end 14, a processing resource 16 and a data interface 18. Theimage sensor 10 is communicatively coupled to a host processor 20. Thehost processor can belong to a variety of different devices, forexample, a tablet, a smart phone or a digital camera. The image sensorarray 12 provides an input to the analogue front end 14, which in turnis connected to the processing resource 16. The data interface 18 isprovided between the processing resource 16 and the host processor 20for supplying image and communication data from the image sensor 10 tothe host processor 20. The image sensor array 12, the analogue front end14 and the processing resource 16 are integrated on a single chip.

The image sensor 10 is configured to receive and process lightcontaining image information and communication information. The imagesensor 10 is configured to operate in two parallel channels: an imagechannel and a communication channel. The image channel is concerned withsensing image signals and outputting image data. The communicationchannel is concerned with sensing communication signals and outputtingcommunication data. The input to both channels is the image sensor array12. The image sensor array 12 is a grid of sensor elements or pixels forreceiving and sensing incident light. The sensor elements may be eitherimage sensors elements for capturing image information or communicationsensor elements for sensing communication signals. The sensor elementscomprise photodiodes or other optical sensing elements. The image sensorelements receive the input light to the image channel and thecommunication sensor elements receive the input light to thecommunication channel. Different suitable arrangement patterns of sensorelements are described with reference to FIG. 2 and FIG. 3 . The imagesensor is constructed by repeating these patterns.

Light reflected or transmitted from an object is suitable for capturingan image of the object, as the intensity of the light received containsimage information about the object and this information may be used toconstruct an image of the object. Colour information is obtained byhaving individual image sensor elements in the image sensor array 12that respond to specific frequencies of light. Each individual imagesensor element converts light into an electrical signal the magnitude ofwhich is proportional to the intensity of the light sensed by the sensorelement. Therefore, at each image sensor element location, the intensityof light of a specific frequency is measured. By collecting intensitiesfrom all image sensor elements, raw image information includinginformation about the intensity of different colours over the imagesensor array 12 is gathered. The image sensor array 12 can containactive pixels, for example it could be a complementarymetal-oxide-semiconductor (CMOS) sensor, where each sensor elementcontains a photodetector, for example a photodiode, and an activeamplifier. Another example of a suitable image sensor array 12 is ahybrid charge-coupled-device (CCD) and CMOS sensor. A hybrid imagesensor array has CCD image sensor elements for image capture and CMOSsensor elements for communication signal capture.

Together with collecting image information, the intended use of theimage sensor 10 is to receive and process light that also includes oneor more components corresponding to a wireless optical communicationsignal sent from a wireless optical communication transmitter. The imagesensor array 12 has dedicated communication sensor elements forcapturing communication information. The communication sensor elementsreceive light for input into the communication channel.

The analogue front end 14 is coupled to the image sensor array 12 andhas two parts, each corresponding to one of the two channels. For theimage channel, the analogue front end 14 has dedicated circuitry forcollecting and conditioning the sensed image signals and for thecommunication channel, the analogue front end 14 has dedicated circuitryfor collecting and conditioning the sensed communication signals. Theanalogue front end 14 contains analogue to digital convertors forconverting the sensed electronic signals into digital signals for theprocessing resource 16. Due to the nature of the incoming optical andhence electronic signals, the properties of the analogue to digitalconvertors may differ between the two parts.

An analogue to digital convertor converts an input analogue signal intoan output digital signal. Two properties of analogue to digitalconvertors relevant to the design and manufacture of the different partsof the analogue front end 14 are: resolution and size. The resolution ofthe convertor is the number of discrete levels the convertor can produceover the range of the input analogue signal. For image electronicsignals, the analogue to digital convertors are chosen such that thedigital resolution of the image meets a pre-determined image quality. Onthe other hand, the analogue to digital convertors for communicationshould be suitable to respond and sample a higher frequency ofelectronic signals stemming from received wireless optical communicationsignals. Values of the frequency of electronic signals may be 100 MHzand above. For example, to deliver bandwidth of 40 MBps, a digital toanalogue convertor is usually configured to sample at least 160 MHz.

The size of an analogue to digital convertor is a further physicalconsideration when manufacturing circuits. Of particular importance hereis the physical area spanned by each analogue to digital convertor. Forcapturing an image, each image sensor element of the image sensor array12 has a corresponding analogue to digital convertor. Alternatively, theimage sensor elements of the image sensor array 12 are grouped intosubgroups each having a dedicated analogue to digital convertor. Ineither case, thousands of analogue to digital convertors need to beimplemented to capture an image and thus the physical limits in size ofthe chip and size of the circuit board limit the choice of suitableanalogue to digital convertors to ones that have a small area. On theother hand, the communication analogue to digital convertors are fewerin number and can therefore have a larger area.

The processing resource 16 has processes digital signals relating to acaptured image and processes digital signals relating to a wirelessoptical communication signal. The processing resource 16 may be twoseparate processors: a first processor dedicated to processing imagesignals and a second processor dedicated to processing communicationsignals.

In use, light containing both information about the image being capturedand an incoming wireless optical communication signal is incident on theimage sensor array 12. As described above, the image sensor 10 operatesin two channels: the image channel and the communication channel. Thesechannels operate simultaneously and in parallel. Firstly, the imagechannel has input sensed by the image sensor elements of the imagesensor array 12 that convert light into electronic signals such that theelectronic signals contain raw image information. The conversion iscarried out by a photodiode. The electronic signals are then provided tothe analogue front end 14. The circuitry of the analogue front end 14collects the electronic image signals and conditions these signals intoa suitable form for the image processor of the processing resource 16.Conditioning includes one or more filtering steps applied to theelectronic signal. These steps may be a low pass filter and a high passfilter. A value for a low pass filter according to an embodiment is 25MHz, e.g. the frequency value below which signals are passed may be 25MHz. A value for a high pass filter according to an embodiment is 300KHz, e.g. the frequency value below which signals are passed may be 300KHz. Conditioning includes converting the analogue electronic imagesignals into digital image signals using at least one dedicated analogueto digital convertor. The image processor of the processing resource 16is then configured to process the digital image signals to produce imagedata suitable for constructing a digital representation of the capturedimage. The image data output of the processing resource 16 is thenprovided to the data interface 18 to be sent to a further processor.

Turning to the communication channel, the image sensor 10 capturesincoming communication signals and communication data is produced. Thecommunication sensor elements of the image sensor array 12 convertincident light into electronic signals such that the electronic signalscontain raw communication information. The electronic signals areprovided to the dedicated communication circuitry of the analogue frontend 14 which collects and conditions the electronic communicationsignals into a suitable form for the processing resource 16.Conditioning can include the optional steps of amplifying the weakelectrical signal induced in the sensor element and equalisation ofreceived signals. Conditioning includes converting the analogueelectronic communication signals into digital communication signalsusing at least one dedicated analogue to digital convertor. The digitalsignal is then passed to the communication part of the processingresource 16 which demodulates the digital signal to extractcommunication data that are encoded in the wireless opticalcommunication signal. Any suitable modulation schemes may be used, forexample non on-off keying modulation schemes are used in someembodiments, and the demodulation is a demodulation from the non on-offkeying modulation scheme. Other non-complex modulation schemes, forexample modulation schemes that do not include or that are not based onreal and imaginary parts, may be implemented in some other embodiments.

The communication data output from the processing resource 16 is thenprovided to the data interface 18. The data interface 18 sendscommunication data output from the communication channel, together withthe image data from the image channel, to the host processor 20. Theinterface and operation of the interface is described in more detailwith reference to FIG. 4 and FIG. 5 .

FIG. 2 shows an example of an arrangement pattern of sensor elementssuitable for the image sensor array 12 of the image sensor 10. The imagesensor array 12 has a grid of sensor elements or pixels for receivingand sensing incident light. Two types of sensor element are shown: animage sensor element and a communication sensor element. Each imagesensor element belongs to the following group: a blue sensor element forsensing blue light, a red sensor element for sensing red light and agreen sensor element for sensing green light. Communication sensorelements that respond to different wavelengths can also be employedusing filters. By introducing a variance in the wavelengths thatdifferent communication pixels respond to, higher throughputs forcommunication may be achieved.

The image sensor array 12 is represented by a rectangular grid with alength of 12 sensor elements and a height of 8 sensor elements. Theimage sensor array 12 is made up of two sensor element arrays: a coloursensor array 22 and a communication sensor array 24. The colour sensorarray 22 has four component image sensor elements: a blue sensor element26, a first green sensor element 28, a second green sensor element 30and a red sensor element 32. Together these four component image sensorelements, arranged is a square of two sensor elements by two sensorelements define a unit cell 34, where the unit cell 34 is the smallest,repeating pattern of the colour sensor array 22. Starting from the lefthand side, the first row of the unit cell 34 is the blue sensor element26 followed by the first green sensor element 28. Again, starting fromthe left hand side, the second row of unit cell 34, below the first row,has the second green sensor element 30 followed by the red sensorelement 32. The unit cell 34 has the first green sensor element 28 andthe second green sensor element 30 in opposite diagonal positions fromeach other. The human eye is more sensitive to green colour than red orblue. By providing two green sensor elements for every one red and oneblue sensor element, the colour sensor array 22 provides an enhancedperceptual signal to noise ratio for a human eye. The colour sensorarray 22 can be drawn by repeating the unit cell 34. The unit cell 34shown in FIG. 2 has the same colour pattern as a Bayer filter.

The pattern of the communication sensor array 24 shown in image sensorarray 12 of FIG. 2 has two rows of 12 sensor elements. Each sensorelement of the communication sensor array 24 is a communication sensorelement 36. The rows of communication sensor elements are arranged atthe periphery, in this case, along the bottom edge of the image sensorarray 12. Although FIG. 2 shows only one such set of rows arranged alongone edge, the communication pixel arrays can be arranged to extend alongtwo edges (either opposite or adjacent), three edges or along all fouredges of the image sensor array 12.

Positioning the communication sensor array 24 at the periphery of theimage sensor array 12 allows for communication signals to be receivedwhile keeping the same perceptual signal to noise ratio for imaging dueto maintaining the number of green sensor elements in the colour sensorarray 22. However, compared to an image sensor array 12 with nocommunication sensor elements, the overall number of image sensorelements is reduced leading to a reduction in number of sensor elementsavailable for an image. Advantageously, no optical artefacts areintroduced by this arrangement.

FIG. 3 shows an alternative sensor element pattern 38 suitable for theimage sensor array 12. The pattern 38 is represented by a rectangulargrid with a length of 12 sensor elements and height of 8 sensorelements. The image sensor array represented by pattern 38 has a coloursensor array made up of image sensor elements and a communication sensorarray made up of communication sensor elements. Groups of one or morecommunication sensor elements of the communication sensor array aredistributed and interspersed within the image sensor elements of thecolour sensor array. These groups are surrounded by image sensorelements. It will be understood that a group of one communication sensorelement refers to a single sensor element.

The pattern 38 of FIG. 3 may be defined by a unit cell 40. The unit cell40 is the smallest, repeating pattern of the pattern 38 and is a squareof two sensor elements by two sensor elements. The component sensorelements of the unit cell 40 are a mixture of colour sensor elements andcommunication sensor elements. Starting from the left hand side, thefirst row of the unit cell 40 is a blue sensor element 42 followed by agreen sensor element 44. Again, starting from the left hand side, thesecond row of unit cell 40, below the first row, is a communicationsensor element 46 followed by a red sensor element 48. The image sensorarray represented by image sensor array 12 can be drawn by repeating theunit cell 40.

The image sensor array represented by pattern 38 has an equal number ofgreen, red, blue and communication sensor elements. This sensor elementarrangement is more suited to applications where pixel resolution ismore important than human perceptual signal to noise ratio, for examplein military imaging applications.

FIG. 4 and FIG. 5 show two schematic diagrams of example implementationsof the data interface 18 between the image sensor 10 and the hostprocessor 20. The data interface 18 may be any high throughput datainterface. In this example, this is a camera serial interface (CSI), forexample CSI-2 or CSI-3. FIG. 4 shows a parallel mode of operation of thedata interface 18. FIG. 4 shows a time axis 50 representing a period oftime. FIG. 4 has two parallel data streams: an image data stream 52 anda communication data stream 54. A first image data packet 56 at a firsttime and a second image data packet 58 at a second time are shown in theimage data stream 52. Communication data 60 are shown on thecommunication data stream 54. Image data from the image channel andcommunication data from the communication channel of the processingresource 16 arrive at the data interface 18 at the same moment in time.The data interface 18 then transmits the two streams in parallel to thehost processor 20.

Arbitration is a process of allocating access to shared resources. Thereis no need for arbitration in parallel mode. Instead, sensorconfiguration information is shared between the host processor 20 andthe image sensor 10 over a separate control bus during a configurationphase. The implementation of the separate control bus is described withreference to FIG. 11 . The sensor configuration information containsknowledge of how bandwidth of the data interface 18 is split between theimage data stream 52 and the communication data stream 54, for example,how many data interface 18 lanes are attributed to communication and howmany lanes are attributed to image. A data interface of the hostprocessor 20 that decodes the transmitted data also has access to thesensor configuration information. Since the sensor configurationinformation contains all the information needed for parallel mode tooperate without any further need for arbitration. The parallel mode issuitable when there is an excess of available throughput in the datainterface 18 protocol. For example, by decreasing image resolution ofthe image sensor array 12 the volume of image data transmitted would bedecreased and thus a parallel mode would be suitable.

FIG. 5 shows a serial mode of operation of the data interface 18. FIG. 5shows a time axis 62 representing a period of time. FIG. 5 has a singledata stream 64. Following the time axis 62, the data stream 64 has afirst communication data packet 66 at a first time, a second image datapacket 68 at a second time and a third communication data packet 70 at athird time. Image data from the image channel and communication datafrom the communication channel of the processing resource 16 aretransmitted by the data interface 18 in a time-multiplexed fashion tothe host processor 20. The image part of the processing resource 16 andthe communication part of the processing resource 16 both need toacquire permission from the host processor 20 before a transmission isinitiated. The data interface 18 merges the incoming image data streamand the incoming communication stream such that the image andcommunication data are transmitted in series. The serial mode ofoperation is applicable when an image stream and a communication streamneed higher bandwidth and cannot be transmitted in parallel. Inparticular, this mode of operation is well suited to applications whereresources are scare, for example where there is a tight power budget ina device or if there is a demand for higher image bandwidth for highquality image capture.

FIG. 6 shows a block diagram of an image sensor 71. The image sensor 71is a modified version of the image sensor 10 to include an additionalflash unit 72. The image sensor 71 has all the components as describedwith reference to FIG. 1 with the addition of the following: a flashprocessing resource 74 and a flash analogue front end 76. The flash unit72 contains one or more LEDs. The flash analogue front end 76 containsdriving circuitry to drive the one or more LEDs of the flash unit 72.The image sensor array 12, the analogue front end 14, the processingresource 16, the flash analogue front end 76 and the flash processingresource 74 are all integrated on to a single chip.

In operation, the host processor 20 provides a communication data signalto the flash processing resource 74 which encodes the communication datasignal onto a drive signal. The drive signal is provided to the flashanalogue front end 76 to produce a drive current for driving the LEDs ofthe flash unit 72. A suitable LED has an operating range in the region 0to 20 MHz. The flash unit 72, once driven, produces an outgoing wirelesscommunication signal. The flash unit 72 can operate either in an imagemode or in a communication mode.

A standard flash unit 72 may contain more than one LED. An alternativeto the above arrangement is to provide more than one different drivingcircuit in the flash analogue front end 76 for the more than one LEDs ofthe flash unit 72. In this arrangement, the flash unit 72 has a firstgroup of image LEDs and a second group of communication LEDs and theimage LEDs are driven by image part of the processing resource 16 andthe flash LEDs are driven by the flash processing resource 74. This modeof operation, allows the flash unit 72 to produce a flash and a wirelessoptical communication signal simultaneously.

FIG. 7 is a schematic diagram of a proximity sensor capable of wirelessoptical light communication. FIG. 7 shows an integrated proximity sensor78 communicatively connected to a host processor 80. The proximitysensor is intended to be used in conjunction with an access point 82that can send and receive optical wireless communication signals. Theproximity sensor 78 has several components integrated on a single chip:an emitter 84, a receiver 86, an optical light communication module 88,a proximity signal module 90 and an ambient light signal module 92. Theproximity sensor 78 is interfaced with the host processor 80 by a firstdata interface 94 between the communication module 88 and the hostprocessor 80. The proximity sensor 78 is also interfaced with the hostprocessor 80 via a second data interface 96. The communication module 88contains a processor and driving circuitry.

The proximity sensor 78 is configured to generate proximity datarepresentative of a nearby object or to make an ambient lightmeasurement. Proximity sensors are commonly found placed next to camerasensors on mobile phone devices and tablets. Proximity data may be usedfor a variety of different purposes. The first data interface 94 is adedicated high throughput interface, for example a camera serialinterface (CSI-2/3). The second data interface 96 is a low throughputinterface, for example a configuration interface that is used for bothcontrol and read-out. The second data interface 96 may be, for example,an inter-integrated circuit (I2C) or serial peripheral interface (SPI).

The proximity sensor operates to perform proximity or ambient lightsensing together with wireless optical communication operation. The hostprocessor 80 sends a signal to the proximity signal module 90 via thesecond data interface 96. The proximity signal module 90 drives theemitter 84 to generate an outgoing optical signal, for example aninfrared signal. The outgoing optical signal strikes a nearby object andis then sensed by and the receiver 86 which senses the incoming opticalsignal. The incoming optical signal is then processed by the proximitysignal module 90 to obtain proximity data. The proximity data is thensent to the host processor 20 via the second data interface 96. In asimilar fashion, the proximity sensor 78 is able to perform an ambientlight measurement by sensing an incoming optical signal at the receiver86 which converts the incoming ambient light into an electrical signal.The sensed electrical signal is then processed by the ambient lightsignal module 92 to obtain ambient light data which may be sent to thehost processor 80 via the second data interface 96.

In addition to the above functionality the proximity sensor 78 isconfigured to emit wireless optical light communication signals.Communication data to be transmitted is sent from the host processor 80to the communication module 88 via the first data interface 94. Thecommunication module 88 receives digital data signal via the first datainterface 94 and has a processor to modulate this communication dataonto a drive current and driving circuitry to provide the drive currentto the emitter 84. The emitter 84 then produces an outgoing modulatedwireless optical communication signal that carries the communicationdata. The signal is then received by the access point 82.

Optionally, the communication module 88 also has a receiver processorand an analogue front end for processing received light. In operation,light containing a wireless optical communication signal is sensed bythe receiver 86 and the receiver 86 converts the incident light into anelectric signal. The analogue front end of the communication module 88collects and conditions this signal, with the optional step ofamplifying it. The analogue front end also include analogue to digitalconvertors to produce a digital signal from the electric signal. Theanalogue to digital convertors are capable of sampling at the frequencyof the wireless optical communication signals. These digital signals arethen processed by the receiver processor. Processing the digital signalsinvolves demodulating them to extract encoded communication data. Theextracted communication data is then sent to the host processor 80 viathe first data interface 94.

Suitable modes of operation of the first data interface 94 and thesecond data interface 96 are described in relation to FIG. 8 and FIG. 9. FIG. 8 shows a parallel mode of operation between the first datainterface 94 and the second data interface 96. FIG. 8 shows a time axis98, a first data stream 100 and a second data stream 102. The first datastream 100 carries communication data 104 from the communication module88. The second data stream 102 carries a first proximity/ambient lightdata packet 106 and a second proximity/ambient light data packet 108from the ambient light signal module 92 and/or proximity signal module90. Parallel mode of operation involves the first data stream 100 to betransmitted over the first data interface 94 and the second data stream102 over the second data interface 96 simultaneously.

FIG. 9 shows a serial mode of operation that can be carried out over thesecond data interface. FIG. 9 shows a time axis 110 and a data stream112. The data stream 112 carries a first communication data packet 114,a proximity data and/or ambient light data packet 116 and a secondcommunication data packet 118. The proximity data and/or ambient lightdata packet 116 is transmitted from the proximity signal module 90and/or the ambient light signal module 92 over the second data interface96. In addition, the first communication data packet 114 and the secondcommunication data packet 118 are transmitted over the second datainterface 96. The first communication data packet 114, secondcommunication data packet 118 and proximity data and/or ambient lightdata packet 116 are transmitted over the second data interface 96 in atime-multiplexed fashion to the host processor 80. The second datainterface 96 merges the incoming communication data and the incomingambient light/proximity data into the data stream 112 such that theimage and communication data are transmitted in series. The serial modeof operation is suitable for low bandwidth applications. For example, ifthe communication data does not require much bandwidth, then it may besent alongside the proximity and/or ambient light data over the lowthroughput second data interface 96. In such a case, the first datainterface 94 may be not used.

FIG. 10 is a top view of an enclosure 120 for a proximity sensor. Theproximity sensor is located inside the enclosure 120. The enclosure 120has a top surface 122 and a side surface 124. On the top surface 122 aretwo holes: a first transmission hole 126 and a first receiving hole 128.On the side surface 124 are two holes: a second transmission hole 130and a second receiving hole 132. The second transmission hole 130 andsecond receiving hole 132 are optional, and the enclosure may beconstructed without them.

As described above, the proximity sensor can produce an outgoing opticalproximity signal and an outgoing wireless optical communication signal.The proximity sensor can receive an incoming optical proximity signaland an incoming wireless optical communication signal. The holes of theenclosure 120 are positioned relative to emitter and receiver of theproximity sensor as follows. The first transmission hole 126 ispositioned such that the outgoing optical proximity signal can beemitted through it. The first receiving hole 128 is positioned such thatthe incoming optical proximity signal can be received through it.

Concerning the optical communication signals FIG. 10 shows twopossibilities for position. As a first option, the proximity sensor mayalso be configured to emit the outgoing optical wireless communicationthrough the first transmission hole 126. For example, proximity sensormay be configured such that the same LED produces both the outgoingproximity signal and the outgoing wireless optical communication signal.Likewise, the proximity sensor may be configured to receive the incomingoptical wireless communication through the first receiving hole 128. Forexample, the proximity sensor may be configured such that the samephotodiode may receive both the incoming proximity signal and theincoming wireless optical communication signal.

Alternatively, the proximity sensor may be configured to emit theoutgoing proximity signal through the first transmission hole 126 andthe outgoing wireless optical communication signal through the secondtransmission hole 130. In this case, the proximity sensor is configuredsuch that a first LED produces the outgoing proximity signal and asecond LED produces the outgoing wireless optical communication signal.Likewise, the proximity sensor may be configured to receive the incomingproximity signal through the first receiving hole 128 and the incomingwireless optical communication signal through the second receiving hole132. In this case, the proximity sensor is configured such that a firstphotodiode receives the incoming proximity signal and a secondphotodiode receives the incoming wireless optical communication signal.

FIG. 11 shows a schematic block diagram showing a wireless opticalcommunication module 134 and a host processor 136. The module 134 has animage sensor 138, as described with reference to FIG. 1 and a proximitysensor 140 as described with reference to FIG. 7 . The image sensor 138of the module 134 is capable of receiving an incoming wireless opticalcommunication signal 142 and an incoming image signal. The proximitysensor 140 is capable of producing an outgoing wireless opticalcommunication signal 144 and an outgoing optical proximity signal.

A configuration bus 146 connects the image sensor 138, the proximitysensor 140 and the host processor 136. The configuration bus 146provides an interface for configuration data to be transmitted from thehost processor 136 to the module 134. The configuration bus is alow-bandwidth communication bus. The configuration bus 146 also providesa data interface between the image sensor 138 and the proximity sensor140 to allow for communication between the two sensors to adjust variousimage capture parameters, for example determining an optimal zoomconfiguration. The configuration bus may be an inter-integrated circuit(I2C) or a serial peripheral interface (SPI).

A first data bus 148 connects the image sensor 138 and the hostprocessor 136. The first data bus 148 carries communication data andimage data transmitted from the image sensor 138. The first data bus 148is a high throughput interface, for example a camera serial interface(CSI-2/3). Compared to a known camera sensor and proximity sensor moduleno physical modification to the first data bus 148 is required. However,unlike a known camera sensor and proximity sensor module, the first databus 148 carries both types of data. The configuration bus 146 can sendconfiguration data including permission data between the image sensor138 and the host processor 136.

A second data bus 150 connects the proximity sensor 140 and the hostprocessor 136. In addition to modifications to sensor hardware, thesecond data bus 150 is added to a known camera sensor and proximitysensor module to carry communication data from the host processor 136 tothe proximity sensor 140 to be transmitted as part of a wireless opticalcommunication signal. The second data bus 150 is a high throughputinterface, for example a camera serial interface (CSI-2/3). Using acamera serial interface (CSI-2/3) provides uniformity between first databus 148 and the second data bus 150.

A skilled person will appreciate that variations of the disclosedarrangements are possible without departing from the invention.Accordingly, the above description of the specific embodiment is made byway of example only and not for the purposes of limitation. It will beclear to the skilled person that in alternative embodimentsstraightforward modifications may be made to each of the featuresdescribed.

The invention claimed is:
 1. A proximity sensor configured to generateproximity data representative of a nearby object using optical signals,the proximity sensor comprising: an emitter configured to produce anoutgoing optical signal comprising at least one of a proximity signal ora modulated communication data signal; a proximity signal moduleconfigured to process an optical proximity signal to obtain proximitydata, wherein the proximity sensor is configured to receive both anincoming optical signal comprising the optical proximity signal and anoptical wireless communication signal modulated in accordance with anoptical wireless communication scheme; and a processing resourceconfigured to perform a demodulation process in respect of at least partof the incoming optical signal to obtain incoming communication data andfurther configured to transmit the incoming communication data and theproximity data in parallel via a first data interface and a second datainterface respectively, wherein the incoming communication data iscarried by a first data stream and is transmitted over the first datainterface and the proximity data is carried by a second data stream andis transmitted over the second data interface, and wherein the seconddata interface is configured to transmit data at a lower rate than thefirst data interface.
 2. The proximity sensor of claim 1, wherein theprocessing resource is further configured to transmit the incomingcommunication data to at least one further processing resource and/or toreceive outgoing communication data from at least one further processingresource.
 3. The proximity sensor of claim 1, wherein the emitter,receiver, and processing resource are integrated on a single chip. 4.The proximity sensor of claim 1, further comprising: driving circuitryconfigured to provide a modulation signal to the emitter, wherein themodulation signal is based on outgoing communication data received fromone further processor.
 5. The proximity sensor of claim 4, wherein thedriving circuitry comprises: a digital to analog convertor configured toprovide a modulation signal at a frequency characteristic of an opticallight communication signal.
 6. The proximity sensor of claim 1, whereinthe proximity sensor further comprises: a receiver sensor configured togenerate a communication signal in response to received light, andconditioning circuitry configured to collect the communication signaland condition the communication signal into a form suitable for theprocessing resource.
 7. The proximity sensor of claim 6, wherein thereceiver sensor comprises: a photodiode configured to receive theproximity signal, and a further photodiode configured to receive theoptical wireless communication signal.
 8. The proximity sensor of claim1, further comprising: an enclosure, wherein the optical proximitysignal is sensed through a first enclosure entrance from a firstdirection, and wherein the optical wireless communication signal issensed through a second enclosure entrance from a second direction thatis different from the first direction.
 9. The proximity sensor of claim1, further comprising: an enclosure, wherein the proximity signal isemitted through a first enclosure exit in a first direction, and whereinthe modulated communication data signal is emitted through a secondenclosure exit in a second direction that is different from the firstdirection.
 10. The proximity sensor of claim 1, wherein the proximitysensor is further configured to perform ambient light sensing.
 11. Theproximity sensor of claim 1, wherein the first data interface comprisesa high speed interface.
 12. The proximity sensor of claim 1, wherein thefirst data interface comprises a camera serial interface (CSI).
 13. Amethod of receiving communication data comprising: receiving a lightsignal by using the proximity sensor of claim 1, where the light signalcomprises an optical wireless communication signal modulated inaccordance with an optical communication protocol; processing anddemodulating at least part of the light signal that was received toobtain the incoming communication data; and transmitting the incomingcommunication data to at least one further processing resource.
 14. Amethod of transmitting or receiving communication data using a proximitysensor configured to generate proximity data representative of a nearbyobject using optical signals, wherein the proximity sensor is configuredto receive both an incoming optical signal comprising an opticalproximity signal and configured to receive an incoming optical signalcomprising an optical wireless communication signal modulated inaccordance with an optical wireless communication scheme; and whereinthe proximity sensor comprises: an emitter configured to produce anoutgoing optical signal comprising at least one of a proximity signal ora modulated communication data signal; and a processing resourceconfigured to perform a demodulation process in respect of at least partof the incoming optical signal to obtain incoming communication data andconfigured to transmit the incoming communication data and the proximitydata in parallel via a first data interface and a second data interfacerespectively, wherein the incoming communication data is carried by afirst data stream and is transmitted over the first data interface andthe proximity data is carried by a second data stream and is transmittedover the second data interface, and wherein the second data interface isconfigured to transmit data at a lower rate than the first datainterface the method comprising at least one of: using an emitter of theproximity sensor to emit light comprising a modulated wirelesscommunication signal; or using a receiver of the proximity sensor toreceive light comprising an optical wireless communication signal;generating an electrical signal by the receiver in response to the lightreceived; and processing, by the processing resource of the proximitysensor, at least part of the electrical signal to obtain incomingcommunication data from the optical wireless communication signal.